Network based storage (or simply, “network storage”) is a common approach to backing up data, making large amounts of data accessible to multiple users, and other purposes. In a network storage environment, a storage server makes data available to client (host) systems by presenting or exporting to the clients one or more logical containers of data. There are various forms of network storage, including network attached storage (NAS) and storage area network (SAN). In a NAS context, a storage server services file-level requests from clients, whereas in a SAN context a storage server services block-level requests. Some storage servers are capable of servicing both file-level requests and block-level requests.
The technology marketplace has been experiencing several trends that impact existing network storage technologies. For example, the number of random input-output operations (IOPS) per-unit-cost of solid-state storage such as flash memory has leapfrogged that of more traditional storage media such as hard disk drives (HDDs). Also, the ratio of spinning media IOPS to capacity for HDDs is diminishing as disk sizes increase. These trends make it increasingly desirable to incorporate solid-state storage into network storage systems. Yet doing so with existing systems can be complicated and may require expensive modifications. In addition, there are indications that faster microprocessor cores will not be achievable indefinitely, and that in the future the emphasis will be more on including more cores per machine than on designing faster cores.
In addition, it is becoming more important in the marketplace for network storage systems to be highly adaptable to different applications and performance profiles. Yet in existing network storage systems, the file system code tends to be tightly coupled with low-level layout code. That tight coupling results in increasing complexity when attempting to adapt a given system to different uses. Conventional network storage systems each generally use a single data layout, which may be adequate (or perhaps better than adequate) for some uses, but generally cannot be optimal for any, much less all, of the myriad storage-related applications that the marketplace requires.
The competitive landscape also increasingly favors the use of storage space efficiency techniques, such as deduplication, and related technologies such as snapshots and cloning. These techniques are all commonly implemented by using block sharing, a technique in which data blocks on storage media are shared by two or more logical data entities (e.g., files) to avoid storing separate identical instances of those blocks. However, the data layout techniques used to achieve block sharing often adversely affect performance, particularly sequential I/O latency, since a shared block typically cannot be in the ideal location for all of the data containers that use it. In effect, block sharing creates data fragmentation. Consequently, a network storage system designer must make a design-time choice about how much weight to give these competing concerns. That choice will then fix the performance characteristics of the system and is extremely difficult if not impossible to change once the design has been implemented and productized.